{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "3cb8c2d2",
   "metadata": {},
   "source": [
    "\n",
    "Copyright(c) 2025 NVIDIA Corporation. All rights reserved.\n",
    "\n",
    "NVIDIA Corporation and its licensors retain all intellectual property\n",
    "and proprietary rights in and to this software, related documentation\n",
    "and any modifications thereto.Any use, reproduction, disclosure or\n",
    "distribution of this software and related documentation without an express\n",
    "license agreement from NVIDIA Corporation is strictly prohibited."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Enable speculative speech processing for the Digitial Human Pipeline. \n",
    "\n",
    "This notebook provides step-by-step instructions for enabling the speculative speech processing for the Digital Human Pipeline. Speculative speech processing enables real-time, natural conversational AI by processing interim transcripts and generating early responses while maintaining conversation coherence. It uses NVIDIA’s specialized frame processors and context aggregators.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "83ac94c6",
   "metadata": {},
   "source": [
    "### Ensure that the Pre-requisites are met\n",
    "\n",
    "Please run through the steps noted in the setup prequisites notebook to ensure that you have the necessary setup in place.\n",
    "\n",
    "Check the status of the application again by running the following command. \n",
    "\n",
    "Note: You will need to select the python environment to run the notebook for the first time. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aad607f8",
   "metadata": {},
   "outputs": [],
   "source": [
    "!kubectl get po -n app"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "718eb463",
   "metadata": {},
   "source": [
    "### Download the ACE Controller Configuration\n",
    "* Ensure that the downloaded ACE Controller configuration is in sync. You can click on the download button again to download the latest version of the configuration. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a5ef241",
   "metadata": {},
   "source": [
    "![ACE Configurator](pictures/ace_configurator_sync.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Update the bot pipeline\n",
    "1. Navigate to the following path in your file system:\n",
    "   ```\n",
    "   ace-controller-ace-controller-deployment -> assets -> app-storage-volume -> src -> bot.py\n",
    "   ```\n",
    "2. Open the `bot.py` file.\n",
    "\n",
    "![Avatar Controller Config](pictures/ace_controller_bot.png)\n",
    "\n",
    "3. The speculative speech processing is disabled by default. You can enable it by uncommenting/commenting the lines as guided in the code. Snippets shown below are from the `bot.py` file. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cb8bf9cd",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pipecat_nvidia.processors.nvidia_context_aggregator import (\n",
    "    create_nvidia_context_aggregator,\n",
    "    # NvidiaTTSResponseCacher, # Uncomment to enable speculative speech processing\n",
    ")\n",
    "\n",
    "\n",
    "# Comment out the below line when enabling Speculative Speech Processing\n",
    "context_aggregator = llm.create_context_aggregator(context)\n",
    "\n",
    "# Uncomment the below line to enable speculative speech processing\n",
    "# nvidia_context_aggregator = create_nvidia_context_aggregator(context, send_interims=True)\n",
    "# Uncomment the below line to enable speculative speech processing\n",
    "# nvidia_tts_response_cacher = NvidiaTTSResponseCacher()\n",
    "\n",
    "\n",
    "pipeline = Pipeline(\n",
    "            [\n",
    "                transport.input(),\n",
    "                user_presence_processor,\n",
    "                stt, # Speech-To-Text\n",
    "                stt_transcript_synchronization,\n",
    "                # Comment out the below line when enabling Speculative Speech Processing\n",
    "                context_aggregator.user(),\n",
    "                # Uncomment the below line to enable speculative speech processing\n",
    "                # nvidia_context_aggregator.user(),\n",
    "                llm,\n",
    "                proactivity_processor,\n",
    "                tts,\n",
    "                # Uncomment the below line to enable speculative speech processing\n",
    "                # nvidia_tts_response_cacher,  # For caching TTS response when Speculative Speech Processing is enabled\n",
    "                tts_transcript_synchronization,\n",
    "                custom_view_processor,\n",
    "                facial_gestures,\n",
    "                a2f,\n",
    "                posture_provider_processor,\n",
    "                animgraph,\n",
    "                # Comment out the below line when enabling Speculative Speech Processing\n",
    "                context_aggregator.assistant(),\n",
    "                # Uncomment the below line to enable speculative speech processing\n",
    "                # nvidia_context_aggregator.assistant(),\n",
    "                transport.output(),  # Websocket output to client\n",
    "            ]\n",
    "        )\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c74eb56b",
   "metadata": {},
   "source": [
    "### Upload the Changes\n",
    "Save your local changes in VSCode. \n",
    "In the ACE Configurator view in VSCode, click the **Upload** button to upload the modified `bot.py` file."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93b4832e",
   "metadata": {},
   "source": [
    "![Avatar Controller Upload Config](pictures/upload_config.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "949cb7de",
   "metadata": {},
   "source": [
    "Ensure that the changes are finished updating. This might take a few seconds.\n",
    "\n",
    "Once the changes are successfully updated. the ACE Configurator view will show a green `Synced` symbol again for the app-storage-volume. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba8e9f9b",
   "metadata": {},
   "source": [
    "![Avatar Controller Upload Config](pictures/ace-configurator-synced.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### Refresh the Browser UI\n",
    "\n",
    "1. Open the browser UI where the digital human is hosted. (your current URL:30111)\n",
    "2. Refresh the page.(Note: you might need to refresh it a couple of times)\n",
    "3. Start a conversation with the digital human.\n",
    "\n",
    "###  Test the speculative speech processing\n",
    "\n",
    "Observe that the digital human responds latency is reduced for user queries.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conclusion\n",
    "You have successfully reconfigured the ACE digital human and tested the changes. 🎉"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
